Multi-class SVMs for Image Classification using Feature Tracking

In this paper a novel representation for image classification is proposed which exploits the temporal information inherent in natural visual input. Image sequences are represented by a set of salient features which are found by tracking of visual features. In the context of a multi-class classification problem this representation is compared against a representation using only raw image data. The dataset consists of image sequences generated from a processed version of the MPI face database. We consider two types of multi-class SVMs and benchmark them against nearest-neighbor classifiers. By introducing a new set of SVM kernel functions we show that the feature representation significantly outperforms the view representation

Psychophysical investigation of facial expressions using computer animated faces

The human face is capable of producing a large variety of facial expressions that supply important information for communication. As was shown in previous studies using unmanipulated video sequences, movements of single regions like mouth, eyes, and eyebrows as well as rigid head motion play a decisive role in the recognition of conversational facial expressions. Here, flexible but at the same time realistic computer animated faces were used to investigate the spatiotemporal coaction of facial movements systematically. For three psychophysical experiments, spatiotemporal properties were manipulated in a highly controlled manner. First, single regions (mouth, eyes, and eyebrows) of a computer animated face performing seven basic facial expressions were selected. These single regions, as well as combinations of these regions, were animated for each of the seven chosen facial expressions. Participants were then asked to recognize these animated expressions in the experiments. The findings show that the animated avatar in general is a useful tool for the investigation of facial expressions, although improvements have to be made to reach a higher recognition accuracy of certain expressions. Furthermore, the results shed light on the importance and interplay of individual facial regions for recognition. With this knowledge the perceptual quality of computer animations can be improved in order to reach a higher level of realism and effectiveness

A similarity-based approach to perceptual feature validation

Which object properties matter most in human perception may well vary according to sensory modality, an important consideration for the design of multimodal interfaces. In this study, we present a similarity-based method for comparing the perceptual importance of object properties across modalities and show how it can also be used to perceptually validate computational measures of object properties. Similarity measures for a set of three-dimensional (3D) objects varying in shape and texture were gathered from humans in two modalities (vision and touch) and derived from a set of standard 2D and 3D computational measures (image and mesh subtraction, object perimeter, curvature, Gabor jet filter responses, and the Visual Difference Predictor (VDP)). Multidimensional scaling (MDS) was then performed on the similarity data to recover configurations of the stimuli in 2D perceptual/computational spaces. These two dimensions corresponded to the two dimensions of variation in the stimulus set: shape and texture. In the human visual space, shape strongly dominated texture. In the human haptic space, shape and texture were weighted roughly equally. Weights varied considerably across subjects in the haptic experiment, indicating that different strategies were used. Maps derived from shape-dominated computational measures provided good fits to the human visual map. No single computational measure provided a satisfactory fit to the map derived from mean human haptic data, though good fits were found for individual subjects; a combination of measures with individually-adjusted weights may be required to model the human haptic similarity judgments. Our method provides a high-level approach to perceptual validation, which can be applied in both unimodal and multimodal interface design

The inaccuracy and insincerity of real faces

Since conversation is a central human activity, the synthesis of proper conversational behavior for Virtual Humans will become a critical issue. Facial expressions represent a critical part of interpersonal communication. Even with the most sophisticated, photo-realistic head model, an avatar who's behavior is unbelievable or even uninterpretable will be an inefficient or possibly counterproductive conversational partner. Synthesizing expressions can be greatly aided by a detailed description of which facial motions are perceptually necessary and sufficient. Here, we recorded eight core expressions from six trained individuals using a method-acting approach. We then psychophysically determined how recognizable and believable those expressions were. The results show that people can identify these expressions quite well, although there is some systematic confusion between particular expressions. The results also show that people found the expressions to be less than convincing. The pattern of confusions and believability ratings demonstrates that there is considerable variation in natural expressions and that even real facial expressions are not always understood or believed. Moreover, the results provide the ground work necessary to begin a more fine-grained analysis of the core components of these expressions. As some initial results from a model-based manipulation of the image sequences shows, a detailed description of facial expressions can be an invaluable aid in the synthesis of unambiguous and believable Virtual Humans

Psychophysics for perception of (in)determinate art

The question of how humans perceive art and how the sensory percept is endowed with aesthetics by the human brain has continued to fascinate psychologists and artists alike. It seems, for example, rather easy for us to classify a work of art as either "abstract" or "representational". The artist Robert Pepperell recently has produced a series of paintings that seek to defy this classification: his goal was to convey "indeterminancy" in these paintings - scenes that at first glance look like they contain an object or belong to a certain genre but that upon closer examination escape a definite determination of their contents. Here, we report results from several psychophysical experiments using these artworks as stimuli, which seek to shed light on the perceptual processing of the degree of abstraction in images. More specifically, the task in these experiments was to categorize a briefly shown image as "abstract" or "representational". Stimuli included Pepperell‘s paintings each of which was paired with a similar representational work of art from several periods and several artistic genres. The results provide insights into the visual processes determining our perception of art and can also function as a "objective" validation for the intentions of the artist

Do congenital prosopagnosia and the other-race effect affect the same face recognition mechanisms?

Congenital prosopagnosia (CP), an innate impairment in recognizing faces, as well as the other-race effect (ORE), a disadvantage in recognizing faces of foreign races, both affect face recognition abilities. Are the same face processing mechanisms affected in both situations? To investigate this question, we tested three groups of 21 participants: German congenital prosopagnosics, South Korean participants and German controls on three different tasks involving faces and objects. First we tested all participants on the Cambridge Face Memory Test in which they had to recognize Caucasian target faces in a 3-alternative-forced-choice task. German controls performed better than Koreans who performed better than prosopagnosics. In the second experiment, participants rated the similarity of Caucasian faces that differed parametrically in either features or second-order relations (configuration). Prosopagnosics were less sensitive to configuration changes than both other groups. In addition, while all groups were more sensitive to changes in features than in configuration, this difference was smaller in Koreans. In the third experiment, participants had to learn exemplars of artificial objects, natural objects, and faces and recognize them among distractors of the same category. Here prosopagnosics performed worse than participants in the other two groups only when they were tested on face stimuli. In sum, Koreans and prosopagnosic participants differed from German controls in different ways in all tests. This suggests that German congenital prosopagnosics perceive Caucasian faces differently than do Korean participants. Importantly, our results suggest that different processing impairments underlie the ORE and CP

Visualization and (Mis)Perceptions in Virtual Reality

Virtual Reality (VR) technologies are now being widely adopted for use in areas as diverse as surgical and military training, architectural design, driving and flight simulation, psychotherapy, and gaming/entertainment. A large range of visual displays (from desktop monitors and head-mounted displays (HMDs) to large projection systems) are all currently being employed where each display technology offers unique advantages as well as disadvantages. In addition to technical considerations involved in choosing a VR interface, it is also critical to consider perceptual and psychophysical factors concerned with visual displays. It is now widely recognized that perceptual judgments of particular spatial properties are different in VR than in the real world. In this paper, we will provide a brief overview of what is currently known about the kinds of perceptual errors that can be observed in virtual environments (VEs). Subsequently we will outline the advantages and disadvantages of particular visual displays by foc using on the perceptual and behavioral constraints that are relevant for each. Overall, the main objective of this paper is to highlight the importance of understanding perceptual issues when evaluating different types of visual simulation in VEs

The Evaluation of Stylized Facial Expressions

Stylized rendering aims to abstract information in an image making it useful not only for artistic but also for visualization purposes. Recent advances in computer graphics techniques have made it possible to render many varieties of stylized imagery efficiently. So far, however, few attempts have been made to characterize the perceptual impact and effectiveness of stylization. In this paper, we report several experiments that evaluate three different stylization techniques in the context of dynamic facial expressions. Going beyond the usual questionnaire approach, the experiments compare the techniques according to several criteria ranging from introspective measures (subjective preference) to task-dependent measures (recognizability, intensity). Our results shed light on how stylization of image contents affects the perception and subjective evaluation of facial expressions

Measuring the Discernability of Virtual Objects in Conventional and Stylized Augmented Reality

In augmented reality, virtual graphical objects are overlaid over the real environment of the observer. Conventional augmented reality systems normally use standard real-time rendering methods for generating the graphical representations of virtual objects. These renderings contain the typical artifacts of computer generated graphics, e.g., aliasing caused by the rasterization process and unrealistic, manually configured illumination models. Due to these artifacts, virtual objects look artifical and can easily be distinguished from the real environment. A different approach to generating augmented reality images is the basis of stylized augmented reality [FBS05c]. Here, similar types of artistic or illustrative stylization are applied to the virtual objects and the camera image of the real enviroment. Therefore, real and virtual image elements look significantly more similar and are less distinguishable from each other. In this paper, we present the results of a psychophysical study on the effectiveness of stylized augmented reality. In this study, a number of participants were asked to decide whether objects shown in images of augmented reality scenes are virtual or real. Conventionally rendered as well as stylized augmented reality images and short video clips were presented to the participants. The correctness of the participants' responses and their reaction times were recorded. The results of our study show that an equalized level of realism is achieved by using stylized augmented reality, i.e., that it is significantly more difficult to distinguish virtual objects from real objects

Visualizing Natural Image Statistics

Natural image statistics is an important area of research in cognitive sciences and computer vision. Visualization of statistical results can help identify clusters and anomalies as well as analyze deviation, distribution and correlation. Furthermore, they can provide visual abstractions and symbolism for categorized data. In this paper, we begin our study of visualization of image statistics by considering visual representations of power spectra, which are commonly used to visualize different categories of images. We show that they convey a limited amount of statistical information about image categories and their support for analytical tasks is ineffective. We then introduce several new visual representations, which convey different or more information about image statistics. We apply ANOVA to the image statistics to help select statistically more meaningful measurements in our design process. A task-based user evaluation was carried out to compare the new visual representations with the conventional power spectra plots. Based on the results of the evaluation, we made further improvement of visualizations by introducing composite visual representations of image statistics